An Empirical Study on Bidirectional Recurrent Neural Networks for Human Motion Recognition

The deep recurrent neural networks (RNNs) and their associated gated neurons, such as Long Short-Term Memory (LSTM) have demonstrated a continued and growing success rates with researches in various sequential data processing applications, especially when applied to speech recognition and language modeling. Despite this, amongst current researches, there are limited studies on the deep RNNs architectures and their effects being applied to other application domains. In this paper, we evaluated the different strategies available to construct bidirectional recurrent neural networks (BRNNs) applying Gated Recurrent Units (GRUs), as well as investigating a reservoir computing RNNs, i.e., Echo state networks (ESN) and a few other conventional machine learning techniques for skeleton-based human motion recognition. The evaluation of tasks focuses on the generalization of different approaches by employing arbitrary untrained viewpoints, combined together with previously unseen subjects. Moreover, we extended the test by lowering the subsampling frame rates to examine the robustness of the algorithms being employed against the varying of movement speed

Uncertainty-aware video visual analytics of tracked moving objects

Vast amounts of video data render manual video analysis useless while recent automatic video analytics techniques suffer from insufficient performance. To alleviate these issues we present a scalable and reliable approach exploiting the visual analytics methodology. This involves the user in the iterative process of exploration hypotheses generation and their verification. Scalability is achieved by interactive filter definitions on trajectory features extracted by the automatic computer vision stage. We establish the interface between user and machine adopting the VideoPerpetuoGram (VPG) for visualization and enable users to provide filter-based relevance feedback. Additionally users are supported in deriving hypotheses by context-sensitive statistical graphics. To allow for reliable decision making we gather uncertainties introduced by the computer vision step communicate these information to users through uncertainty visualization and grant fuzzy hypothesis formulation to interact with the machine. Finally we demonstrate the effectiveness of our approach by the video analysis mini challenge which was part of the IEEE Symposium on Visual Analytics Science and Technology 2009

Visual Analytics of Gaze Data with Standard Multimedia Player

With the increasing number of studies, where participants’ eye movements are tracked while watching videos, the volume of gaze data records is growing tremendously. Unfortunately, in most cases, such data are collected in separate files in custom-made or proprietary data formats. These data are difficult to access even for experts and effectively inaccessible for non-experts. Normally expensive or custom-made software is necessary for their analysis. We address this problem by using existing multimedia container formats for distributing and archiving eye-tracking and gaze data bundled with the stimuli data. We define an exchange format that can be interpreted by standard multimedia players and can be streamed via the Internet. We convert several gaze data sets into our format, demonstrating the feasibility of our approach and allowing to visualize these data with standard multimedia players. We also introduce two VLC player add-ons, allowing for further visual analytics. We discuss the benefit of gaze data in a multimedia container and explain possible visual analytics approaches based on our implementations, converted datasets, and first user interviews

Artificial Neural Networks for Automated Quality Control of Textile Seams

Bahlmann C, Heidemann G, Ritter H. Artificial Neural Networks for Automated Quality Control of Textile Seams. Pattern Recognition. 1999;32(6):1049-1060.We present a method for an automated quality control of textile seams, which is aimed to establish a standardized quality measure and to lower coals in manufacturing. The system consists of a suitable image acquisition setup, an algorithm for locating the seam, a feature extraction stage and a neural network of the self-organizing map type for feature classification. A procedure to select an optimized feature set carrying the information relevant for classification is described. (C) 1999 Pattern Recognition Society. Published by Elsevier Science Ltd, All rights reserved

Interactive Image Data Labeling Using Self-Organizing Maps in an Augmented Reality Scenario

Bekel H, Heidemann G, Ritter H. Interactive Image Data Labeling Using Self-Organizing Maps in an Augmented Reality Scenario. Neural Networks. 2005;18(2005 Special Iss.):566-574.We present an approach for the convenient labeling of image patches gathered from an unrestricted environment. The system is employed for a mobile Augmented Reality (AR) gear: While the user walks around with the head-mounted AR-gear, context-free modules for focus-of-attention permanently sample the most “interesting” image patches. After this acquisition phase, a Self-Organizing Map (SOM) is trained on the complete set of patches, using combinations of MPEG-7 features as a data representation. The SOM allows visualization of the sampled patches and an easy manual sorting into categories. With very little effort, the user can compose a training set for a classifier, thus, unknown objects can be made known to the system. We evaluate the system for COIL-imagery and demonstrate that a user can reach satisfying categorization within few steps, even for image data sampled from walking in an office environment

Fronts between Hopf- and Turing-type domains in a two-component reaction-diffusion system

Propagating and standing fronts between Hopf- and Turing-type domains are observed experimentally on a one-dimensional array of resistively coupled nonlinear LC-oscillators describable by a two-component reaction-diffusion equation. Numerical and experimental results are compared in particular with respect to front velocities. In the neighbourhood of a codimension-two point two coupled Ginzburg-Landau equations, derived by multiple scale methods, are useful approximation

Recognition of Gestural Object Reference with Auditory Feedback

Bax I, Bekel H, Heidemann G. Recognition of Gestural Object Reference with Auditory Feedback. In: Kaynak O, ed. Artificial neural networks and neural information processing. Proceedings. Lecture notes in computer science. Vol 2712. Berlin: Springer; 2003: 425-432.We present a cognitively motivated vision architecture for the evaluation of pointing gestures. The system views a scene of several structured objects and a pointing human hand. A neural classifier gives an estimation of the pointing direction, then the object correspondence is established using a sub-symbolic representation of both the scene and the pointing direction. The system achieves high robustness because the result (the indicated location) does not primarily depend on the accuracy of the pointing direction classification. Instead, the scene is analysed for low level saliency features to restrict the set of all possible pointing locations to a subset of highly likely locations. This transformation of the "continuous" to a "discrete" pointing problem simultaneously facilitates an auditory feedback whenever the object reference changes, which leads to a significantly improved human-machine interaction

A Hierarchical Feed-forward Network for Object Detection Tasks

Bax I, Heidemann G, Ritter H. A Hierarchical Feed-forward Network for Object Detection Tasks. In: Szu HH, ed. Independent component analyses, wavelets, unsupervised smart sensors, and neural networks III. Proceedings. SPIE, International Society for Optical Engineering. Vol 5818. Bellingham, Wash.: SPIE; 2005: 144-152

Interactive Auditory Display to Support Situational Awareness in Video Surveillance

Presented at the 17th International Conference on Auditory Display (ICAD2011), 20-23 June, 2011 in Budapest, Hungary.A key element for efficient video surveillance is situational awareness. Characteristics of human perception (e.g., inattentional blindness) as well as surveillance practice (e.g., CCTV operators have multiple responsibilities) often hinder comprehensive visual recognition of the activities in the monitored area. We support sit- uational awareness and reduce the workload of CCTV operators by complementing the video display by an auditory display. Tra- jectories of moving objects extracted from surveillance video are sonified by auditory icons. These icons are interactively assigned by the user to each object category of the video and, in this way, form a sonic ecology. We use a spatial auditory display to rep- resent location, direction and velocity of a trajectory with respect to a virtual listener. This facilitates orientation in virtual auditory space in a natural and realistic way that meets users’ expectations. Modification areas are introduced to allow the users to define areas in which auditory icons are modified to further improve situational awareness. We put emphasis on efficient interaction between users and the auditory display to adjust the system according to the mon- itored area. Finally, we evaluate our approach by a user study and discuss benefits and shortcomings of the proposed sonification in the light of psychology, cognitive science, and neuroscience

Adaptive Computer Vision: Online Learning for Object Recognition

Bekel H, Bax I, Heidemann G, Ritter H. Adaptive Computer Vision: Online Learning for Object Recognition. In: Rasmussen CE, Bülthoff HH, Giese MA, Schölkopf B, eds. Pattern Recognition. Proceedings. Lecture notes in computer science. Vol 3175. Berlin: Springer-Verlag; 2004: 447-454.The "life" of most neural vision systems splits into a one-time training phase and an application phase during which knowledge is no longer acquired. This is both technically inflexible and cognitively unsatisfying. Here we propose an appearance based vision system for object recognition which can be adapted online, both to acquire visual knowledge about new objects and to correct erroneous classification. The system works in an office scenario, acquisition of object knowledge is triggered by hand gestures. The neural classifier offers two ways of training: Firstly, the new samples can be added immediately to the classifier to obtain a running system at once, though at the cost of reduced classification performance. Secondly, a parallel processing branch adapts the classification system thoroughly to the enlarged image domain and loads the new classifier to the running system when ready